fix: OOM on cold parse (V8 SlicedString retention) + kiro projectPath for repo attribution - #984
Open
Enclavet wants to merge 2 commits into
Open
fix: OOM on cold parse (V8 SlicedString retention) + kiro projectPath for repo attribution#984Enclavet wants to merge 2 commits into
Enclavet wants to merge 2 commits into
Conversation
… cache String.prototype.slice returns a V8 SlicedString: a view that retains a reference to its ENTIRE parent string. The parsers store short previews of message text (userMessage.slice(0, 500/2000)) in the long-lived session cache. Session files routinely carry 100KB+ strings (agent- injected system prompts, tool results), so every cached preview pinned its full parent buffer for the life of the process. Measured on 3.2GB of kiro CLI session files (6,659 files, largest 40MB): cold parse, default heap, before: 4.33GB peak -> OOM crash cold parse, 8GB heap, before: 5.67GB peak (kiro provider alone) after kiro flatSlice: 0.91GB peak after parser.ts cache sites too: 0.64GB peak original failing command (cold, default heap, all providers): 0.89GB peak -> completes Warm runs were always fine (~0.29GB) because the cache's JSON round-trip flattens the strings on load — which made this bug appear intermittent: it only fired on a cold or invalidated cache. Fix: flatSlice() in content-utils.ts forces a flat copy via Buffer round-trip. Applied at the six kiro userMessage capture sites and the three shared cache-building sites in parser.ts (protects all providers). Regression test asserts the no-retention property via bounded heap growth over 1000 large-parent slices. AI-Origin: human
The kiro provider was reading the full working directory from session metadata (meta.cwd for CLI sessions, workspacePaths[0] for v2 IDE sessions, workspaceDirectory for workspace sessions) but discarding it via basename(), keeping only the leaf name for display. This meant computeAttributionRecords could never resolve kiro sessions to a git repo, so `codeburn sync push --attribution` produced 0 facts for all kiro-originated sessions. Now passes the full path as projectPath on emitted ParsedProviderCalls, which buildRepoGroups uses to resolve git identity and correlate commits with sessions via timestamp windows. The path stays local: only the normalized origin remote egresses in attribution spans. Behavior changes beyond attribution: - kiro calls now flow through canonicalizeProviderCallProject, so kiro sessions in LINKED GIT WORKTREES canonicalize to the main repository: their report project name changes from the worktree dir name to the main repo name (consistent with claude/codex behavior). - workingDirectory is now populated on kiro calls. - PROVIDER_PARSE_VERSIONS.kiro bumped (project-path-v1): cached entries predate projectPath and are served without re-invoking the parser, so without the bump this fix silently no-ops for every warm cache. The bump forces a one-time cold kiro re-parse on upgrade. ORDERING: this commit must land WITH (or after) the preceding SlicedString OOM fix. The forced cold re-parse it triggers is exactly the workload that OOM'd before that fix on multi-GB kiro stores. Perf: per-call canonicalization added a measured +5% to cold parse (.git-marker lstat walk per call). resolveCanonicalProjectPath is now memoized on cwd (cleared with the session cache), removing the redundant walks for all providers. Tests: projectPath emission fixtures for all three session formats (CLI, v2 IDE, workspace-session), fingerprint-change assertion, and a regression test seeding a pre-bump cache entry and proving the re-parse recovers projectPath. AI-Origin: human
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Two related fixes for the kiro provider and shared parser, ordered deliberately (see "Landing order" below):
fix(parser): cold-parse OOM caused by V8 SlicedString retention in the session cachefix(kiro): emit the session's full working directory asprojectPathsocodeburn yieldandsync push --attribution(feat(sync): opt-in git attribution spans onsync push --attribution#848) can resolve kiro sessions to their git repos1. The OOM
codeburn sync push --since=all(and any cold parse of a large kiro store) crashed withFATAL ERROR: Ineffective mark-compacts near heap limitat V8's default ~4GB heap.Root cause:
String.prototype.slicereturns a V8 SlicedString — a view that retains areference to its entire parent string. The parsers store short previews of message text
(
userMessage.slice(0, 500/2000)) in the long-lived session cache. Kiro CLI session entriesroutinely carry 100KB+ strings (agent-injected system prompts, tool results), so every cached
preview pinned its full parent buffer for the life of the process.
Measured on 3.2GB of kiro CLI session files (6,659 files, largest 40MB):
flatSliceat kiro capture sitesWarm runs were always fine (~0.29GB) because the cache's JSON round-trip flattens strings on
load — which made the bug look intermittent: it only fired on a cold or invalidated cache.
Fix:
flatSlice(s, max)(bounded prefix as a flat copy via Buffer round-trip) andflatString(s)(unconditional flat copy, for short strings that are views — regex matchgroups and
trim()results). Applied at the six kirouserMessagecapture sites, the threeshared cache-build sites in
parser.ts(protects all providers), the title/agentTypetrim().slice()sites, andextractToolNames(regex match groups retain the entire scannedcontent buffer).
2. kiro
projectPathThe kiro provider read the full working directory from session metadata (
meta.cwdfor CLIsessions,
workspacePaths[0]for v2 IDE sessions,workspaceDirectoryfor workspacesessions) but discarded it via
basename(), keeping only the leaf name for display. Repogrouping in
yield.tstherefore never resolved kiro sessions to a git repo —codeburn yieldmisgrouped them under the cwd fallback, and
sync push --attributionproduced 0 facts for allkiro sessions. On my machine this fix took a 6-month attribution window from 59 to 318 facts.
Behavior changes to be aware of:
report under the main repository's project name (consistent with claude/codex)
workingDirectoryis now populated on kiro callsPROVIDER_PARSE_VERSIONS.kirois bumped (project-path-v1): cached entries predateprojectPathand are served without re-invoking the parser, so without the bump this fixsilently no-ops for every warm cache. Upgrading triggers a one-time cold kiro re-parse.
resolveCanonicalProjectPathis now memoized on cwd (kiro's per-call canonicalizationmeasured ~+5% cold parse before memoization; the same cache benefits all providers)
Landing order (important)
The version bump forces a cold kiro re-parse on upgrade — exactly the workload that OOM'd
before commit 1. These commits must land together, in this order. Please don't split or
reorder them.
Tests
tests/flat-slice.test.ts: prefix/identity/multi-byte behavior, a pinned decision formid-surrogate-pair cuts (U+FFFD), and heap-growth property tests proving no parent retention
for both
flatSliceand regex-matchflatStringtests/kiro-projectpath.test.ts:projectPathemission fixtures for all three kiro sessionformats (CLI, v2 IDE, workspace-session), a fingerprint-change assertion, and a regression
test seeding a pre-bump cache entry and proving the re-parse recovers
projectPathtscclean; kiro/sync/yield/parser suites pass (178 tests)